3D Convolutional Neural Networks for Tumor Segmentation using Long-range 2D Context

We present an efficient deep learning approach for the challenging task of tumor segmentation in multisequence MR images. In recent years, Convolutional Neural Networks (CNN) have achieved state-of-the-art performances in a large variety of recognition tasks in medical imaging. Because of the considerable computational cost of CNNs, large volumes such as MRI are typically processed by subvolumes, for instance slices (axial, coronal, sagittal) or small 3D patches. In this paper we introduce a CNN-based model which efficiently combines the advantages of the short-range 3D context and the long-range 2D context. To overcome the limitations of specific choices of neural network architectures, we also propose to merge outputs of several cascaded 2D-3D models by a voxelwise voting strategy. Furthermore, we propose a network architecture in which the different MR sequences are processed by separate subnetworks in order to be more robust to the problem of missing MR sequences. Finally, a simple and efficient algorithm for training large CNN models is introduced. We evaluate our method on the public benchmark of the BRATS 2017 challenge on the task of multiclass segmentation of malignant brain tumors. Our method achieves good performances and produces accurate segmentations with median Dice scores of 0.918 (whole tumor), 0.883 (tumor core) and 0.854 (enhancing core). Our approach can be naturally applied to various tasks involving segmentation of lesions or organs.Comment: Submitted to the journal Computerized Medical Imaging and Graphic

Atlas encoding by randomized forests for efficient label propagation

Abstract We propose a method for multi-atlas label propagation based on encoding the individual atlases by randomized classification forests. Most current approaches perform a non-linear registration between all atlases and the target image, followed by a sophisticated fusion scheme. While these approaches can achieve high accuracy, in general they do so at high computational cost. This negatively affects the scalability to large databases and experimentation. To tackle this issue, we propose to use a small and deep classification forest to encode each atlas individually in reference to an aligned probabilistic atlas, resulting in an Atlas Forest (AF). At test time, each AF yields a probabilistic label estimate, and fusion is done by averaging. Our scheme performs only one registration per target image, achieves good results with a simple fusion scheme, and allows for efficient experimentation. In contrast to standard forest schemes, incorporation of new scans is possible without retraining, and target-specific selection of atlases remains possible. The evaluation on three different databases shows accuracy at the level of the state of the art, at a significantly lower runtime

Spectral Forests: Learning of Surface Data, Application to Cortical Parcellation

International audienceThis paper presents a new method for classifying surface datavia spectral representations of shapes. Our approach benefits classificationproblems that involve data living on surfaces, such as in cortical parcellation.For instance, current methods for labeling cortical points into surface parcelsoften involve a slow mesh deformation toward pre-labeled atlases, requiringas much as 4 hours with the established FreeSurfer. This may burden neurosciencestudies involving region-specific measurements. Learning techniquesoffer an attractive computational advantage, however, their representation ofspatial information, typically defined in a Euclidean domain, may be inadequatefor cortical parcellation. Indeed, cortical data resides on surfaces thatare highly variable in space and shape. Consequently, Euclidean representationsof surface data may be inconsistent across individuals. We proposeto fundamentally change the spatial representation of surface data, by exploitingspectral coordinates derived from the Laplacian eigenfunctions ofshapes. They have the advantage over Euclidean coordinates, to be geometryaware and to parameterize surfaces explicitly. This change of paradigm,from Euclidean to spectral representations, enables a classifier to be applieddirectly on surface data via spectral coordinates. In this paper, we decide tobuild upon the successful Random Decision Forests algorithm and improve itsspatial representation with spectral features. Our method, Spectral Forests,is shown to significantly improve the accuracy of cortical parcellations overstandard Random Decision Forests (74% versus 28% Dice overlaps), and produceaccuracy equivalent to FreeSurfer in a fraction of its time (23 secondsversus 3 to 4 hours)

Deep Learning with Mixed Supervision for Brain Tumor Segmentation

International audienceMost of the current state-of-the-art methods for tumor segmentation are based on machine learning models trained on manually segmented images. This type of training data is particularly costly, as manual delineation of tumors is not only time-consuming but also requires medical expertise. On the other hand, images with a provided global label (indicating presence or absence of a tumor) are less informative but can be obtained at a substantially lower cost. In this paper, we propose to use both types of training data (fully-annotated and weakly-annotated) to train a deep learning model for segmentation. The idea of our approach is to extend segmentation networks with an additional branch performing image-level classification. The model is jointly trained for segmentation and classification tasks in order to exploit information contained in weakly-annotated images while preventing the network to learn features which are irrelevant for the segmentation task. We evaluate our method on the challenging task of brain tumor seg-mentation in Magnetic Resonance images from BRATS 2018 challenge. We show that the proposed approach provides a significant improvement of seg-mentation performance compared to the standard supervised learning. The observed improvement is proportional to the ratio between weakly-annotated and fully-annotated images available for training

Technical Appendix on Sparse Bayesian Regression

Technical appendix to a MICCAI 2014 conference paper. Loïc Le Folgoc, Hervé Delingette, Antonio Criminisi, and Nicholas Ayache: Sparse Bayesian RegistrationInternational audienceWe report the technical details for a sparse bayesian approach to regression. It can be seen as an extension of the Relevance Vector Machine of Tipping to a more general setting that can handle vector-valued regression and generic quadratic priors

Recognizing cardiac magnetic resonance acquisition planes

International audienceIn this paper we propose a method for automatic wrangling of missing or noisy acquisition plane information of cardiac magnetic resonance images in order to simplify case filtering and image lookup in large collections of cardiac data. To recognize standard cardiac planes we use features based on image miniatures combined with a decision forest classifier. We show that augmenting the dataset with a set of nondestructive transformations can improve classification accuracy. Our approach compares favorably to the state of the art while requiring fewer manual annotations

Deep Neural Decision Forests

Abstract We present Deep Neural Decision Forests -a novel approach that unifies classification trees with the representation learning functionality known from deep convolutional networks, by training them in an end-to-end manner. To combine these two worlds, we introduce a stochastic and differentiable decision tree model, which steers the representation learning usually conducted in the initial layers of a (deep) convolutional network. Our model differs from conventional deep networks because a decision forest provides the final predictions and it differs from conventional decision forests since we propose a principled, joint and global optimization of split and leaf node parameters. We show experimental results on benchmark machine learning datasets like MNIST and ImageNet and find onpar or superior results when compared to state-of-the-art deep models. Most remarkably, we obtain Top5-Errors of only 7.84%/6.38% on ImageNet validation data when integrating our forests in a single-crop, single/seven model GoogLeNet architecture, respectively. Thus, even without any form of training data set augmentation we are improving on the 6.67% error obtained by the best GoogLeNet architecture (7 models, 144 crops)

Quantifying Registration Uncertainty with Sparse Bayesian Modelling

International audienceWe investigate uncertainty quantification under a sparse Bayesian model of medical image registration. Bayesian modelling has proven powerful to automate the tuning of registration hyperparameters, such as the trade-off between the data and regularization functionals. Sparsity-inducing priors have recently been used to render the parametrization itself adaptive and data-driven. The sparse prior on transformation parameters effectively favors the use of coarse basis functions to capture the global trends in the visible motion while finer, highly localized bases are introduced only in the presence of coherent image information and motion. In earlier work, approximate inference under the sparse Bayesian model was tackled in an efficient Variational Bayes (VB) framework. In this paper we are interested in the theoretical and empirical quality of uncertainty estimates derived under this approximate scheme vs. under the exact model. We implement an (asymptotically) exact inference scheme based on reversible jump Markov Chain Monte Carlo (MCMC) sampling to characterize the posterior distribution of the transformation and compare the predictions of the VB and MCMC based methods. The true posterior distribution under the sparse Bayesian model is found to be meaningful: orders of magnitude for the estimated uncertainty are quantitatively reasonable, the uncertainty is higher in textureless regions and lower in the direction of strong intensity gradients

Sparse Bayesian Registration

International audienceWe propose a Sparse Bayesian framework for non-rigid registration. Our principled approach is flexible, in that it efficiently finds an optimal, sparse model to represent deformations among any preset, widely over-complete range of basis functions. It addresses open challenges in state-of-the-art registration, such as the automatic joint estimate of model parameters (e.g. noise and regularization levels). We demonstrate the feasibility and performance of our approach on cine MR, tagged MR and $3$D US cardiac images, and show state-of-the-art results on benchmark data sets evaluating accuracy of motion and strain